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Lll: Entry 1 of 7 



File: USPT 



Oct 9, 



2001 



DOCUMENT-IDENTIFIER: US 6301303 Bl 

TITLE: Method and apparatus for predictively coding shape information of video 
signal 



Application Filing Date (1) : 
19970519 

Brief Summary Text (3) : 

The present invention relates in general to coding a video signal in a desired 
unit, and more particularly to a method and an apparatus for predictively coding 
shape information of the video signal, which compare shape information of the 
current image with that of the previous image to obtain a difference therebetween 
and code the shape information of the current image only when the obtained 
difference exceeds a predetermined reference value. For example, the present 
invention is applicable to a shape information coding method of the moving picture 
experts group-4 (referred to hereinafter as MPEG - 4) which is an international 
standard on moving pictures and audio coding, and other image coding methods 
considering shape information. 

Brief Summary Text (5) : 

Conventionally, in coding a moving picture in the unit of object, shape information 
are transmitted together with motion information beginning with that having the 
highest priority, for the prediction of motion compensation. At this time, 
different motion information, must be applied to adjacent pixels on the object 



Brief Summary Text (11) : 

Recently, ISO/IEC/WGll has considered a method of coding an object with arbitrary 
shape information, differently from MPEG -1 and MPEG - 2 performing frame-unit coding. 

Brief Summary Text (12): 

Here, a given video is divided into a background image and an object image, and a 
rectangle including the divided background image and object image is defined as a 
video object plane (referred to hereinafter as VOP) . In MPEG - 4, in the case where 
object regions including desired objects or areas are present in images, they are 
divided into VOPs and the divided VOPs are coded individually. 

Brief Summary Text (32): 

The shape information is a video object plane . 
Detailed Description Text (9): 

FIG. 4 is a block diagram illustrating the construction of each of the VOP coders 
12A, 12B, . . . , 12N in the encoder 10 in FIG. 1. As shown in this drawing, each 
VOP coder includes a motion estimator 31, a motion compensator 32, a subtracter 33, 
a texture coder 34, an adder 35, a previous VOP detector 36, a shape coder 37, a 
multiplexer 38 and a buffer 39. 

Detailed Description Text (18) : 



boundary . 
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Noticeably, an output signal from the shape coder 37 is variably used according to 
fields to which the VOP coders 12A, 12B, . . . , 12N are applied. As indicated by 
the dotted lines in the drawing, the output signal from the shape coder 37 may be 
provided to the motion estimator 31, the motion compensator 32 and the texture 
coder 34 to be used for the motion estimation and compensation and the texture 
information coding. 

Detailed Description Text (19) : 

The motion information estimated by the motion estimator 31, the texture 
information coded by the texture coder 34 and the shape information coded by the 
shape coder 37 are multiplexed by the multiplexer 38 and transmitted through the 
buffer 39 to the multiplexer 13 in FIG. 1 for the transmission in the form of bit 
stream. 

Detailed Description Text (20) : 

In MPEG-4, various shape coding techniques can be applied to the shape coder 37 
which codes shape information of each VOP from the VOP formation unit 11. For 
example, one such technique may be an MMR shape coding technique which codes shape 
information on the basis of N. times. N blocks, where N=16, 8 or 4. Other shape 
coding techniques may be a vertex-based shape coding technique, a baseline-based 
shape coding technique, a context-based arithmetic coding technique, etc. 

Detailed Description Text (22): 

First, the unit image for coding (UIC) will be described. In conventional image 
coding standards such as MPEG -1 and MPEG - 2, there has been proposed a block-based 
coding technique for coding a video signal in the unit of frame, macro block or 
unit block regardless of the contents of image. However, as the requirements of 
multimedia function are increased, the necessity for video signal coding around 
object is increased, as well. Such an object-around coding technique requires the 
prescription of a new coding unit around object, not a fixed coding unit such as 
the conventional frame or block unit. In other words, the original frame-unit video 
signal is divided into significant object images with shape information, and the 
coding is performed with respect to each of the divided object images as a coding 
unit. For example, in MPEG -4, the VOP is defined as the unit image for coding. 

Other Reference Publication (3) : 

VM Action Group, " MPEG -4 Video Verification Model Version 1.0 (Report ISO/IEC 
JTC1/SC29/WG11 MPEG4/N1172) Jan. 1996, ISO/IEC XP00204798 p. 3, Paragraph 1 --p. 
10, Paragraph 1; figures 2.1.1, 3.1.2. 

CLAIMS : 

13. A method for predictively coding shape information of a video signal, as set 
forth in claim 1, wherein the shape information is a video object plane . 
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L21: Entry 5 of 20 



File: USPT 



Jul 13, 1999 



DOCUMENT-IDENTIFIER: US 5923814 A 

TITLE: Methods and apparatus for performing video data reduction operations and for 
concealing the visual effects of data reduction operations 

Application Filing Date (1) : / 
19970815 \/ 

Brief Summary Text (5) : 

One digital video compression and data transmission format that offers particular 
promise with regard to high definition television ("HDTV") is the ISO -MPEG 
(International Standards Organization — Moving Picture Experts Group) standard 
described in a report titled "Coding of Moving Pictures and Associated Audio for 
Digital Storage Media up to about 1.5 Mbits/s", ISO 2 11172 rev 1, Jun. 10, 1992 
hereby expressly incorporated by reference. 

Brief Summary Text (7): 

It is to be understood that various features of the present invention, such as data 
recording techniques, as opposed to, e.g., specific data prioritization and 
selection techniques, are generally not data format dependent and are therefore not 
limited to applications involving specific data formats. 

Detailed Description Text (22) : 

In accordance with this codeword prioritization approach, each codeword in the 'full 
rate data bit stream received by the VTR is assigned a priority level or number 
based on its relative importance to generating a video frame having good image" 
quality. Accordingly, codewords are prioritized as a function of their importance 
in generating video frames during long play modes of VTR playback operation. 

Detailed Description Text (23) : 

The prioritization scheme used to support long play modes of VTR operation are 
likely to be similar to those used for trick play prioritization described in U.S. 
patent application Ser. No. 08/003,887, titled "DIGITAL VIDEO RECORDING DEVICE". 
Accordingly, a digital VTR that contains hardware for trick play prioritization 
such as the one in the above identified copending patent application, could 
implement a long play prioritization scheme according to the present invention at 
little or no additional cost. 

Detailed Description Text (24) : 

One suitable prioritization scheme for prioritizing video codewords for use in a 
reduced rate bit stream suitable for supporting long play mode digital VTR 
operation is described below with regard to an MPEG based data stream. Listed below 
are the types of data that may be contained in a codeword, and the suggested 
priority number to be assigned to the codeword containing the data. It should be 
noted that the data in each codeword correspond to, or is associated with, a 
particular video frame represented by the data in the video data stream. 

Detailed Description Text (25) : 

One suitable prioritization order for long play mode VTR operation, listed in order 
from the most important to the least important data, is as follows: 
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Detailed Description Text (36) : 

While the goal of the above prioritization scheme is to provide a method by which 
the data rate can be reduced, it is important to note that the reduced rate data 
stream generated during VTR long play mode operation should include sufficient data 
to support the same frame display rate supported during standard play mode. 

Detailed Description Text (38) : 

Thus, while the above prioritization scheme is similar to that suggested for trick 
play prioritization, the reduced rate data stream generated for long play mode 
operation should include B-picture data that are of little or no use during trick 
play operation. Such B-pictures are normally not displayed during trick play 
operation and the data needed to generate such B-pictures are therefore normally 
omitted from a trick play data stream. 

Detailed Description Text (42) : 

In such an embodiment, it is desirable to maintain the same relative amounts of 
higher order I-, P- and B- DCT coefficients in the reduced rate bit stream as found 
in the full rate bit stream. Accordingly, the relative amounts of data represented 
by the codewords which are assigned to priority levels 8, 9 and 10, in accordance 
with the prioritization scheme of the present invention, should be selected for 
inclusion in the reduced rate bit stream in approximately the same ratios found in 
the full rate bit stream received by the digital VTR. 

Detailed Description Text (44); 

While the above prioritization scheme is described using MPEG terminology, it 
should be noted that the prioritization scheme can be readily generalized to 
numerous other digital video compression systems. 

Detailed Description Text (51) ; 

Simulations have shown that requantizing with a higher quantization factor gives 
better results than data reduction through the use of data prioritization, when the 
two methods are used to achieve the same total data rate reduction. However, data 
prioritization is a simpler operation to implement than requantization . 
Accordingly, it may be cheaper to implement a long play mode in a digital VTR using 
data prioritization and selection rather than requantization to achieve the desired 
reduction in the data rate. However, the use of requantization provides superior 
results in terms of image quality. 

Detailed Description Text (53) : 

In the previously described methods, the bit rate was reduced by either 1) 
performing variable length decoding, data prioritization, selection of the 
prioritized data and re-encoding of the selected data or 2) performing variable 
length decoding, re-quantizing of the decoded data with an adjusted quantization 
scale factor selected to produce a reduced bit rate, and re-encoding of the re- 
quantized data. In accordance with each of these methods data reduction is 
performed uniformly over each video picture. Accordingly, data reduction is 
performed without taking into consideration the screen location of the image 
portion to be produced by the data upon which data reduction is being performed. 

Detailed Description Text (56) : 

Thus, because of this type of prioritization, when data are selected to form the 
reduced rate bit stream, a relatively high amount of data corresponding to the 
center of video pictures will be selected while a relatively lesser amount of the 
data corresponding to the edges of video pictures will be selected for inclusion in 
the reduced rate bit stream. In this manner, data representing the center portion 
of video pictures may be included in the reduced bit rate stream at the same or at 
a slightly reduced bit rate when compared to the full bit rate stream. However, 
data representing the edges of video pictures will be included in the reduced rate 
bit stream at a bit rate that is significantly lower than the bit rate of such data 
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in the full bit rate data stream. 
Detailed Description Text (64): 

Data reduction methods 1 and 3 can be combined such that data prioritization and 
selection can be used as the method of reducing the data rate to produce a reduced 
rate bit stream from a full rate bit stream. 
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L3: Entry 1 of 1 



File: USPT 



Nov 4, 



2003 



DOCUMENT-IDENTIFIER: US 6643258 Bl 

TITLE: Communication resource management method and node device using priority 
control and admission control 



Application Filing Date (1) : 
19981022 

Brief Summary Text (15) : 

For ins'tance, in the IETF, the scheme called " Differential Service (D. Clark and J. 
Wroclawski, "An Approach to Service Allocation in the Internet ", Internet Draft 
draft-clark-diff-svc-alloc-OO.txt) and the scheme called "SIMA" (K. Kilkki "Simple 
Integrated Media Access (SIMA)", Internet Draft draf t-kalevi-simple-media-access- 
Ol.txt) have been proposed. These are schemes in which a tag indicating a high 
priority level is written into a packet at a rate proportional to the bandwidth 
allocated to the flow. 

Brief Summary Text (16) : 

For example, in the case of allocating a bandwidth of 100 [Kbps] to a flow A, the 
edge node provided at the ingress side of the network monitors the flow A, and 
writes a value indicating a high priority level in packets up to 100 [Kbps] . At a 
node inside the network, when congestion occurs, packets with a lower priority 
level is much more likely to be discarded than packet with higher priority level . 
In this way, the packets belonging to the flow A will be transferred at a high 
priority level within the range of 100 [Kbps] . 

Brief Summary Text (17) ; 

Thus the flow can receive the transfer service at the high priority level as much 
as the bandwidth that are allocated to it. In this scheme, it suffices for the node 
inside the network to deal with the priority level written in the packets without 
becoming conscious of the flow so that there is an advantage that it is sufficient 
to carry out a simple processing. 

Brief Summary Text (18) : 

FIG. 1 shows an exemplary realization of the differential service. In FIG. 1, a 
flow A and a flow B are transferred through a network 301. This network 301 
comprises core nodes 101-103 and edge nodes 201-204, which are connected through 
communication links. The core nodes 101-103 carry out the priority processing at a 
time of packet transfer according to priority tags written in packets. The edge 
nodes 201-204 are located outside the core nodes 101-103 and write tags indicating 
the priority levels with respect to packets of the respective flows. 

Brief Summary Text (19) : 

Within the network 301, the flow A is transferred through a route of the edge node 
204, the core node 103, the core node 102, and the edge node 202, while the flow B 
is transferred through a route of the edge node 201, the core node 101, the core 
node 102, and the edge node 202. The edge node 204 writes a tag indicating the high 
priorit y level into packets up to 100 [Kbps] among the packets belonging to the 
flow A, while the edge node 201 writes a tag indicating the high priority level 
into packets up to 200 [kbps] among the packets belonging to the flow B. At the 
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nodes 101-103 and 201-203 inside the network 301, when it is inevitable to discard 
some packets, the low priority packets, i.e., those packets in which the high 
priority tag is not written, will be discarded at a high priority. As a result, in 
this network 301, the packets belonging to the flow A up to 100 [Kbps] are 
transferred at the high priority while the packets belonging to the flow B up to 
200 [Kbps] are transferred at the high priority. 

Brief Summary Text (25) : 

On the other hand, in the scheme such as the differential service in which a tag 
indicating the priority level is to be written in packets at a rate proportional to 
the bandwidth at an edge node provided at the ingress side of the network and the 
priority control is to be carried out at a time of the transfer processing 
according to this priority level at a node inside the network, there arise the 
second problem in that it is impossible to guarantee the communication quality to 
each flow, and the third problem that it is difficult to make up a plan for 
augmenting the node facilities according to the bandwidth consumed at each node. 

Brief Summary Text (29) : 

According to one aspect of the present invention there is provided a method for 
managing communication resources in a network containing edge nodes located at a 
boundary of the network and core nodes located inside the network, comprising the 
steps of: (a) storing at one edge node an information for obtaining an available 
amount of communication resources that can be newly allocated in the network to one 
set of flows which share at least a route from said one edge node to an egress node 
of the network; (b) carrying out an admission control at said one edge node by 
newly receiving a request for allocation of communication resources for one flow 
belonging to said one set of flows, judging whether or not to accept the request 
according to a requested amount of communication resources and the available amount 
of communication resources as obtained from the information stored at the step (a) 
for said one set of flows, and allocating requested communication resources to said 
one flow when it is judged that the request is to be accepted; and (c) transmitting 
packets at said one edge node by describing a priority level in each packet 
according to an amount of communication resources allocated to a flow of the 
packets at the step (b) , such that a core node carries out a transfer processing 
with respect to received packets according to the priority level described in each 
received packet. 

Brief Summary Text (32) : 

According to another aspect of the present invention there is provided a method for 
managing communication resources in a network containing edge nodes located at a 
boundary of the network and core nodes located inside the network, comprising the 
steps of: (a) carrying out a priority control in which an edge node transmits 
packets by describing a priority level in each packet according to an amount of 
communication resources allocated to a flow of the packets, and a core node carries 
out a transfer processing with respect to received packets according to the 
priority level described in each received packet; (b) transmitting from one edge 
node a notification regarding an amount of communication resources allocated to a 
set of flows which include one flow and which share at least a route from said one 
edge node to an egress node of the network, according to an amount of communication 
resources allocated to said one flow; and (c) receiving the notification at one 
core node and calculating an amount of communication resources to be consumed at 
said one core node according to the notification. 

Brief Summary Text (36) : 

According to another aspect of the present invention there is provided a method for 
managing communication resources in a network containing edge nodes located at a 
boundary of the network and core nodes located inside the network, comprising the 
steps of: (a) carrying out a priority control in which an edge node transmits 
packets by describing a priority level in each packet according to an amount of 
communication resources allocated to a flow of the packets, and a core node carries 



http://westbrs:9000/bin/gate.exe?f=TOC8i&state=qm5k87.4J&ESNAME=KWIC&HT^ 



10/7/04 



Record Display Form 



Page 1 of 1 



Previous Doc Next Doc Go to Doc# 
First Hit 



= 









L6: Entry 1 of 20 



File: PGPB 



Oct 24, 2002 



DOCUMENT- IDENTIFIER: US 20020154210 Al 
TITLE: VIDEOCONFERENCING HARDWARE 



Application Filing Date : 
19970407 

Detail Description Paragraph : 

[0064] Note in FIG. 3 that Data LAN hub 25, A/V Switching Circuitry 30 and MLAN 
Server 60 also provide respective lines 25b, 30b, and 60e for coupling to 
additional multimedia resources 16 (FIG. 1), such as multimedia document 
management, multimedia databases, radio/TV channels, etc. Data LAN hub 25 (via 
bridges/routers 11 in FIG. 1) and A/V Switching Circuitry 30 additionally provide 
lines 25c and 30c for coupling to one or more other MLTUMs 10 which may be in the 
same locality (i.e., not far enough away to require use of WAN technology). Where 
WANs are required, WAN gateways 40 are used to provide highest quality compression 
methods and standards in a shared resource fashion, thus minimizing costs at the 
workstation for a given WAN quality level, as discussed below. 

Detail Description Paragraph : 

[0135] The AVNM 63 manages A/V Switching Circuitry 30 in FIG. 3 for selectively 
routing audio/video signals to and from CMWs 12, and also to and from WAN gateway 
40, as called for by clients. Audio/video devices (e.g., CMWs 12, conference 
bridges 35, multimedia resources 16 and WAN gateway 40 in FIG. 3) connected to A/V 
Switching Circuitry 30 in FIG. 3, have physical connections for audio in, audio 
out, video in and video out. For each device on the network, the AVNM combines 
these four connections into a port abstraction, wherein each port represents an 
addressable bidirectional audio/video channel. Each device connected to the network 
has at least one port. Different ports may share the same physical connections on 
the switch. For example, a conference bridge may typically have four ports (for 
2. times. 2 mosaicing) that share the same video-out connection. Not all devices need 
both video and audio connections at a port. For example, a TV tuner port needs only 
incoming audio/video connections. 
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DOCUMENT-IDENTIFIER: US '6237025 Bl 
TITLE: Multimedia collaboration system 



Application Filing Date (1) : 
19971219 

Detailed Description Text (25) : 

Note in FIG. 3 that Data LAN hub 25, A/V Switching Circuitry 30 and MLAN Server 60 
also provide respective lines 25b, 30b, and 60e for coupling to additional 
multimedia resources 16 (FIG. 1), such as multimedia document management, 
multimedia databases, radio/TV channel setc Data LAN hub 25 (via bridges/routers 11 
in FIG, 1) and A/V Switching Circuitry 30 additionally provide lines 25c and 30c 
for coupling to one or more other MLANs 10 which may be in the same locality (i.e., 
not far enough away to require use of WAN technology) Where WANs are required, WAN 
gateways 40 are used to provide highest quality compression methods and standards 
in a shared resource fashion, thus minimizing costs at the workstation for a given 
WAN quality level, as discussed below. 

Detailed Description Text (101) : 

The AVNM 63 manages A/V Switching Circuitry 30 in FIG. 3 for selectively routing 
audio/video signals to and from CMWs 12, and also to and from WAN gateway 40, as 
called for by clients Audio/video devices (e.g., CMWs 12, conference bridges 35, 
multimedia resources 16 and WAN gateway 40 in FIG. 3) connected to A/V Switching 
Circuitry 30 in FIG. 3, have physical connections for audio in, audio out, video in 
and video out. For each device on the network, the AVNM combines these four 
connections into a port abstraction, wherein each port represents an addressable 
bidirectional audio/video channel. Each device connected to the network has at 
least one port Different ports may share the same physical connections on the 
switch. For example, a conference bridge may typically have four ports (for 
2. times. 2 mosaicing) that share the same video-out connection. Not all devices need 
both video and audio connections at a port. For example, a TV tuner port needs only 
incoming audio/video connections. 
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File: USPT 



Nov 21, 



2000 



DOCUMENT-IDENTIFIER: US 6151598 A 

TITLE: Digital dictionary with a communication system for the creating, updating, 
editing, storing, maintaining, referencing, and managing the digital dictionary 

Application Filing Date (1) : 
19971204 

Detailed Description Text (31) : 

Thirdly, there is Multimedia Reference Pointer 706 means which produces a selective 
multimedia representation or reference material for selective instances of said 
combination of alphabet, code, character, sign, or number, wherein each said 
instance correspond with selective user or application during a selective period of 
time. Furthermore, said multimedia representation can be retrieved from External 
Multimedia Equipment 712 which further includes selective multimedia database, 
transmission, storage, processing, display, or printing machines. 

Detailed Description Text (34) : 

In a more preferred embodiment, FIG. 4 teaches a method for interconnecting a 
selective plurality of document and multimedia equipment for efficient storage and 
retrieval, wherein said document equipment 615 includes a plurality of selective 
document database, transmission, storage, processing, display, or printing 
machines; and said multimedia equipment 712 includes selective multimedia database, 
transmission, storage, processing, display, or printing machines. 

Detailed Description Text (211) : 

For compound document including motion video, the FORM 210 is comprised of a OBJECT 
ID circuit, which receives standard MACROBLOCK subimages from PREP 202 and produce 
a list of moving objects for each input frame. The FORM 210 is also comprised of a 
PRIORITY ASSIGN circuit which evaluate application requirement from OIF 224 and 
communication constraint from BAND 22 8 and assign the appropriate run-time priority 
level to each moving object and MACROBLOCK subimage. Provided the run- time 
bandwidth is insufficient, the FORM 210 is further comprised of an X-Y INTERPOLATE 
circuit which receive MACROBLOCK subimages, moving objects, and their associated 
run-time priority levels and produce a reformatted MACROBLOCK subimage and moving 
objects. When the run-time bandwidth is proven to be sufficient, the FORM 210 is 
further comprised of DATA TRANSFER circuit which transfer the formatted macroblock 
subimage to ENC 208 and PACK 218 for further processing. 

Detailed Description Text (219) : 

The OIF 224, as part of the PACK 218, is comprised of a INTERFACE circuit, which 
receives application priority requirements from user, application, and/or network 
and produce the appropriate message signal to FORM 210 in further assigning the 
run-time priority levels for each object and subimage. OIF 224 is further comprised 
of DRIVER circuit which allow direct interface with a plurality of external 
commercial DOS, UNIX, WINDOW, NT, or 0S2 application programs. OIF 224 is also 
comprised of a programmable BIOS circuit which direct execute under a commercial 
operating system. 
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File: USPT 



Aug 12, 2003 



DOCUMENT-IDENTIFIER: VS 6606413 Bl 

TITLE: Compression packaged image transmission for telemicroscopy 

Application Filing Date (1) : 
19990601 

Brief Summary Text (7) : 

The enabling tool for providing telepathology services is a telemicroscopy system 
connected to a bi-directional telecommunications network which is pervasive enough 
to allow the necessary equipment to be set up and operated virtually anywhere. 
Conventional forms of telemicroscopy equipment are generally well known in the art 
and suitably comprise a remote controlled microscope system where microscope images 
are acquired with a conventional video camera and transmitted, for display, to a 
control system. Remote operation of the microscope system and remote display of 
transmitted images can be realistically performed using a variety of communications 
technologies. However, in order to ensure a general availability of a developing 
telepathology network, interconnectivity is most realistic in the context of narrow 
band or broadband landline connections. Narrow band systems (like PSTN and ISDN) 
generally guarantee worldwide availability for very low costs, but at the price of 
bandwidth and/or channel capacity. Broadband systems (like ATM) allow enhanced 
channel capacity but still suffer from a lack of sufficient bandwidth to allow 
video transmissions at anything approximating real-time. Because of these 
limitations, conventional telemicroscopy systems have had to make certain 
compromises between channel capacity and image quality. The higher the quality of 
the transmitted image, the longer the time it takes to complete a transmission. 
Conversely, when transmission speed is an overriding concern, image quality 
necessarily suffers. 

Brief Summary Text (8) : 

With a limited bandwidth available on PSTN (Public Switched Telephone Network) and 
ISDN (Integrated Services Digital Network), the only means available to increase 
transmission speed is to reduce the average number of transmitted bits per image, 
i.e., compress the digital image data developed by video camera. Before 
telepathology services become truly viable, image transmission must operate at bit 
rates of only a few hundred kilobits or a few megabits per second, which can only 
be achieved through rather large compression of the data. 

Brief Summary Text (9) : 

Most sensory signals contain a substantial amount of redundant or superfluous 
information. For example, a conventional video camera, that captures approximately 
30 frames per second from a stationary image, produces very similar frames, one 
after the other. Compression techniques attempt to remove the superfluous 
information from repetitive frames, such that a single frame can be represented by 
a reduced amount of finite data, or in the case of time varying images, by a lower 
data rate. It is well known in the art that digitized video signals comprise a 
significant amount of statistical redundancy, i.e., samples are similar to each 
other such that one sample can be predicted fairly accurately from another. By 
removing the predictable or similarity component from a stream of samples, the 
video data rate can be reduced. Such statistical redundancy is able to be removed 
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without perturbing the remaining information. That is, the original uncompressed 
data is able to be recovered almost exactly by various inverse operations. The 
algorithms used in a compression system depend on the available bandwidth, the 
features required by the application, and the af f ordability of the hardware 
required for implementation of the compression algorithm on both the encoding and 
decoding side. 

Drawing Description Text { 6) : 

FIG, 4 is a semi-schematic simplified block diagram of one embodiment of a video 
image transmission system including multi-tiered compression according to the 
present invention; 

Drawing Description Text (7) : 

FIG. 5 is a semi-schematic simplified block diagram of a decompression system 
adapted to receive compression packaged is video image transmissions in accordance 
with the invention; 

Detailed Description Text (2) : 

In order to gain a complete understanding of the compression packaged image 
transmission system and method of the present invention, it will be useful to 
examine how the system might function in the context of a typical telepathology 
procedure. During the course of a surgical intervention, as a surgeon is preparing 
to perform an invasive procedure, the surgeon will typically remove a sample of 
tissue from a patient and forward the tissue sample to the hospital's diagnostic 
laboratory for immediate evaluation. The tissue sample is prepared in conventional 
fashion and loaded onto the sample stage of an examining microscope, comprising the 
laboratory's telemicroscopy system, where an image of the tissue sample is captured 
by a video camera and electronically communicated to a pathologist at a remote site 
for evaluation. During the initial, or preliminary, evaluation, a pathologist is 
able to view the tissue sample in macro and is further able to give directions to 
laboratory personnel as to location and direction of sectioning to be performed in 
order to further any subsequent diagnosis. Frozen sections are prepared according 
to the guidelines of the pathologist, the sections are mounted on glass slides, 
appropriately stained, and subsequently loaded onto a robotically controlled 
microscope stage of the telemicroscopy system. Control of the telemicroscopy system 
is then given to the remote-site pathologist. 

Detailed Description Text (4) : 

Following the pathologist's investigation, the pathologist can prepare a report 
which provides the surgeon with his diagnostic opinion in the case, either over the 
same bi-directional communication medium used to examine the specimen, or by any 
one of a number of various other electronic communication forms available to the 
pathologist. In particular, the pathologist may transmit a written protocol to the 
surgeon by facsimile, e-mail, and the like, or make a direct oral report to the 
surgeon and follow-up with written documentation communicated electronically. After 
making a diagnosis, the pathologist releases the telemicroscopy connection and is 
then available for a next consultation with some other surgery team which might be 
preparing to perform a different surgical procedure in a totally different 
location. In the meantime, the original consulting surgeon is able to continue the 
intervention procedure according to the outcome of the diagnosis made by the 
pathologist. As a check, the original consulting surgeon may decide to forward the 
original tissue specimen to a local pathology laboratory for a final diagnosis, in 
accordance, with conventional accepted procedure. This manner of "Gold Standard" 
cross-checking, is particularly useful as a means of acquiring data on the accuracy 
of results obtained by formulating diagnostic opinions on the basis of televised 
images which have been compressed, transmitted over long distances over relatively 
"noisy" communication connections, decompressed and viewed on a high resolution 
video monitor screen. 

Detailed Description Text (6) : 
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Turning now to FIG. 1, there is depicted a simplified semi -schematic block diagram 
of an exemplary host or server telemicroscopy system useful in the practice of the 
present invention. FIG. 1 illustrates the primary components of a remote, or 
robotically, controllable telemicroscope, operable under software program control 
which would be hosted on a control processor such a small platform personal 
computer system. In accordance with the invention, telemicroscopy equipment 
connected to a telecommunication net in accord with a bi-directional communication 
protocol, forms a key enabling tool for establishing effective telemedicine 
services. This allows the system to combine the high-resolution and color 
saturation is integrity of digital, still video images with, the ability to 
establish bi-directional communication between the "server" and a "client" system 
to enable remote, client control of the microscope in real time. Acquisition and 
transmission of high-resolution video images of desired portions of a specimen can 
be performed in a time period consistent with hands-on, real-time optical practice. 

Detailed Description Text (12) : 

If the system is configured to operate semi-closed-loop, or open-loop, stage 
control is not dependent on feedback per se, but it is at least necessary to 
precisely define where the motors controlling the stage were told to go. For 
reasons detailed further below, the transmitted video image is compression packaged 
in a manner that takes stage motion (both relative degree and absolute magnitude) 
into account when defining the compression packaging technique used under various 
circumstances . 

Detailed Description Text (14) : 

A video camera 22 is optically coupled to the microscope to capture diagnostic- 
quality images of microscopic tissue samples disposed on the sample stage. The 
video camera 22 is preferably a high resolution, color, digital video camera 
operating at an image resolution in accordance with, at least, the NTSE 
standardized composite color video image specification. Examples of video cameras 
suitable for use in connection with the present invention include the Sony DKC-5000 
series of video cameras, the Ikegami 370-M video camera, and other makes and models 
of composite, color video cameras of comparable quality and resolution. Images 
captured by the video camera 22 are directed through video image processors 24 
whereby they are able to be displayed on a high resolution digital display screen 
coupled to the telemicroscope control processor 20. 

Detailed Description Text (15) : 

In addition, and in accordance with the present invention, high-resolution video 
images captured by the camera are compression packaged by the control processor 20 
for transmission over a telecommunications interface 26, coupled, in turn, to a 
wide-area-network 28 such as the Internet. 

Detailed Description Text (16) : 

The images developed by the video camera should have at least the resolution 
available under the NTSC standard. In North America and Japan, the NTSC color video 
image comprises approximately 480 pels per image scan line in the red, green and 
blue (RGB) color components. Approximately 480 scan lines comprise an image frame, 
and image frames are generated at a rate of approximately 30 frames per second. If 
each color component is coded as an 8-bit value (24 bits/pel, i.e., "true color") 
representing some form of luminance plus color difference encoding, for example, 
the representing continuous composite video is produced at a bit rate of about 168 
Megabits per second (Mbps) . For truly high resolution image production suitable for 
telepathology applications, the video camera should be able to generate still 
images with resolutions preferably in the range of from 800 . times . 600 
(pels . times . lines) to about 1024 . times , 1024 (pels . times . lines ) . These images are 
transmitted at frame rates of 30 fps and, at 24 bits per pel, this results in an 
image bit rate in excess 750 Mbps. 
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File: USPT 



Aug 20, 1996 



DOCUMENT-IDENTIFIER: US 5548^1532^/ 

TITLE: Apparatus and method for formulating an interactive TV signal 




Application Filing Date (1) : 
19940428 

Brief Summary Text { 5 ) : 

The present invention is directed toward formatting executable codes and data, 
defining interactive applications, with video and audio program material, for 
reliable and convenient access. The method includes compressing audio and video 
programs according to a protocol such as MPEG . The compressed audio and video (AN) 
programs are then segmented into transmission or transport packets,' with audio 
transport packets identified by a first service identifier SCID.sub.Ai, and video 
transport packets identified- by a second' service identifier, SCID.sub.Vi. 
Interactive application programs associated with A/V programs are compiled into 
functional modules and condensed. A module may be executable software or data. A 
module includes code/data and a header. The code/data portion of each module 
undergoes cyclic redundancy coding (CRC) over the entire module and CRC check bits 
are concatenated or appended to the module. Respective modules are segmented into 
transport packets. The code/data transport packets are identified by a third 
service identifier SCID.sub.Di. The transport packets are formed into transmission 
units comprising an integer number of code/data transport packets and a header 
transport packet. A special directory module is generated which includes 
information for informing a receiver device of the modules incorporated in an 
application program. Video packets, audio packets and module packets are 
subsequently time division multiplexed for transmission. 

Detailed Description Text (19) ; 

Regarding the prioritization of packet multiplexing during the non-video packet 
multiplexing time slots, if a priority is impose d.' it is preferred that the signal 
component with packets that occur very infrequently be assigned a higher 
multiplexing priority. * . - , 
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File: USPT 



Jan 28, 2003 



DOCUMENT-IDENTIFIER: US 6512793 Bl 

TITLE: Data processing apparatus and method 



Application Filing Date ( 1) : 
19990426 

Brief Summary Text (7 ) : 

Further, in a moving image, a method to perform encoding in object units has been 
studied as an international standard method, MPEG4 (Moving Picture Experts Group 
phase 4) (Eto, "MPEG4 Standardization" (The Journal of The Institute of Image 
Electronics Engineers of Japan, vol. 25, No. 3, 1996, pp. 223-228) . FIG. 1 shows an 
example of a frame of a moving image to be encoded by the MPEG4 coding. In FIG. 1, 
a frame 20 comprises four objects as shown in FIG, 2, i.e., a background object 28, 
an object 21 representing a helicopter, an object 22 representing a train, and an 
object 23 representing a car. To indicate the shapes of the objects except the 
background, each object is masked such that a black part of a rectangular area 
surrounding the object is an "outer area", and a white part is an "inner area" (24 
to 26 in FIG, 2), and by this masking, an arbitrary shaped object can be handled. 

Brief Summary Text (12) : 

The visual object data has visual_ob j ect_start_code (Visual Object SC in FIG, 25) 
for identification at its header, then prof ile_and_level_indication (PLP in FIG. 
25) indicative of an^encoding level. Tlien, information on visual objects, 
is_visual_object_identifier^ (IVOI in FIG. 25), visual_object_varid (VOVID in FIG. 
25), viausl object priority (VOPRI in FIG. 25), visual_obj ect_type (VOTYPE in FIG. 
25) and the like follow. These data construct header information of the visual 
object. "VOTYPE" has a value "0001" if the image is a moving image obtained by 
image pickup. Then, video object (VO) data as a cluster of moving image code data 
follows , 

Brief Summary Text (13) : 

The VO data is code data indicative of each object. The VO data has 
video_object_start_code (VOSC in FIG. 25) for identification at its header, 
further, the VO data has video object layer data -(VOL data in FIG, 25) to realize 
scalability. The VOL data has video__ob j ect_layer_start_code (VOLSC in FIG. 25) and 
video object plane data (VOP data in FIG. 25) corresponding to one frame of moving 
image. The VOL data has video_ob j ect_layer_width (VOL_width in FIG. 25) and 
video_object_layer_height (VOL_height in FIG. 25) indicative of size, at its 
header. Also, the VOP data has video object plane width {VOP_width in FIG, 25) and 
video object plane height (VOP_height in FIG. 25) indicative of size, at its 
header. Further, the header of the VOL data has bit_rate code indicative of bit 
rate. 

Brief Summary Text (22) : 

The above-described profile and level are indicated in the PLI in a bit stream of 
MPEG4 code data as shown in FIG. 25. Accordingly, a decoder which decodes a bit 
stream of MPEG code data can determine whether or not decoding is possible by 
referring to the PLI, The decoding is impossible in the following case. 
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Brief Summary Text (30) : 

According to the present invention, the foregoing object is attained by providing a 
data processing apparatus having decoding means for decoding code encoded in image 
object units, said apparatus comprising: detection means for detecting the number 
of objects included in input code and the number of objects decodable by said 
decoding means; and control means for controlling the number of objects of the 
input code, based on the number of objects and the number of decodable objects 
detected by said detection means . 

Brief Summary Text (32) : 

According to the present invention, to attain the foregoing object, the above- 
described apparatus further comprises: extraction means for extracting location 
information of the objects included in said code; and combining means for combining 
code of a plurality of objects, based on an instruction from said control means and 
the location information extracted by said extraction means. 

Brief Summary Text (33) : 

Further, to attain the foregoing object, the above-described apparatus further 
comprises: extraction means for extracting motion information indicative of motions 
of the objects included in said code; and combining means for combining a plurality 
of objects based on an instruction from said control means and the motion 
information extracted by said extraction. means . 

Brief Summary Text (35) : 

Further, another object of the present invention is to provide data processing 
apparatus and method which control the number of objects included in code data. 

Brief Summary Text (36) : 

According to the present invention, the foregoing objects are attained by providing 
a data processing apparatus for processing a data array to reproduce an image with 
a plurality of coded image objects, said apparatus comprising: detection means for 
detecting the number of image objects included in said data array; and control 
means for controlling the number of image objects included in said data array based 
on the number of image objects detected by said detection means. 

Brief Summary Text (40) ; 

Further, another object of the present invention is to provide data processing 
apparatus and method which control the number of objects included in code data 
and/or the information amount of the code data. 

Brief Summary Text (41) : 

According to the present invention, the foregoing objects are attained by providing 
a data processing apparatus for processing a data array to reproduce one frame 
image with a plurality of coded image objects, said apparatus comprising: input 
means for inputting a plurality of data arrays; instruction means for instructing 
synthesizing of a plurality of data arrays inputted by said input means; 
designation means for designating coding specifications of a processed data array; 
control means for controlling information amounts of the plurality of data arrays 
inputted by said input means, based on the coding specifications designated by said 
designation means; and synthesizing means for synthesizing the plurality of data 
arrays with information amounts controlled by said control means, based on the 
coding specifications designated by said designation means. 

Drawing Description Text (15) : 

FIG. 12 is an example of 1-frame code data to be motion -compensated; 
Detailed Description Text (9) : 

FIG. 8 is an example of 1-frame code data. The code data has "Header" indicative of 
the attribute of the frame at its head, next, code data indicative of background 
object (Object 0 in FIG. 8). Then, code data of the respective objects, i.e., code 
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File: USPT 



Jul 18, 2000 



DOCUMENT-IDENTIFIER: US 6092107 A 

TITLE; System and method for interfacing MPEG -coded audiovisual objects permitting 
adaptive control 

Abstract Text (1) : 

The invention provides a system and method allowing the adaptation of a nonadaptive 
system for playing/browsing coded audiovisual objects, such as the parametric 
system of MPEG - 4 . The system of the invention is referred to as the programmatic 
system, and incorporates adaptivity on top of the parametric system. The parametric 
system of MPEG - 4 consists of a Systems Demultiplex (Demux) overseen by digital 
media integration framework (DMIF) , scene graph and media decoders, buffers, 
compositer and renderer. Adaptations possible with the invention include interfaces 
in the categories of media decoding, user functionalities and authoring, thus 
allowing a number of enhanced functionalities in response to use input as well as 
graceful degradation in response to limited system resources. The invention 
includes a specification of an interfacing method in the form of an application 
programming interface (API). Hot object, directional, trick mode, transparency and 
other interfaces are specified. 

Application Filing Date (1) : 
19980407 

Parent Case Text (2) : 

This application is related to U.S. Provisional Application Ser. No. 60/042,798 
from which priority is claimed. 

Brief Summary Text (7) : 

The need for interoperability, guaranteed quality and performance and economies of 
scale in chip design, as well as the cost involved in content generation for a 
multiplicity of formats has lead to advances in standardization in the areas of 
multimedia coding, packetization and robust delivery. In particular, ISO MPEG 
(International Standards Organization Motion Picture Experts Group) has 
standardized bitstream syntax and decoding semantics for coded multimedia in the 
form of two standards referred to as MPEG-1 and MPEG- 2 . MPEG -1 was primarily 
intended for use on digital storage media (DSM) such as compact disks (CDs), 
whereas MPEG - 2 was primarily intended for use in a broadcast environment (transport 
stream), although it also supports an MPEG -1 like mechanism for use on DSM (program 
stream) . MPEG - 2 also included additional features such as DSM Command and Control 
for basic user interaction as may be needed for standardized playback of MPEG-2, 
either standalone or networked. 

Brief Summary Text (8) : 

With the advent of inexpensive boards/PCMCIA cards and with availability of Central 
Processing Units (CPUs), the MPEG -1 standard is becoming commonly available for 
playback of movies and games on PCs. The MPEG - 2 standard on the other hand, since 
it addresses relatively higher quality applications, is becoming common for 
entertainment applications via digital satellite TV, digital cable and Digital 
Versatile Disk (DVD) . Besides the 
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Brief Smnmary Text (9) : 

applications and platforms noted, MPEG -1 and MPEG - 2 are expected to be utilized in 
various other configurations, in streams communicated over network and streams 
stored over hard disks/CDs, as well as in the combination of networked and local 
access . 



Brief Sximmary Text (10) : 

The success of MPEG -1 and MPEG-2, the bandwidth limitation of Internet and mobile 
channels, the flexibility of web-based data access using browsers, and the 
increasing need for interactive personal communication has opened up new paradigms 
for multimedia usage and control . In response, ISO -MPEG started work on a new 
standard, MPEG - 4 , The MPEG -4 standard has addressed coding of audio-visual 
information in the form of individual objects and a system for composition and 
synchronized playback of these objects. While the MPEG -4 development of such a 
fixed parametric system continues, in the meantime, new paradigms in communication, 
software and networking such as that offered by the Java language have offered new 
opportunities for flexibility, adaptivity and user interaction. 

Brief Summary Text (13) : 

Second, regardless of the bandwidth available, the client side PC on which decoding 
may have to take place may be limited in CPU and/or memory resources, and 
furthermore, these resources may be time-varying. Third, a multimedia user 
(consumer) may require highly interactive nonlinear browsing and playback; this is 
not unusual, since a lot of textual content on web pages is capable of being 
browsed using hyperlinked features and the same paradigm is expected for 
presentations employing coded audio-visual objects. The parametric MPEG-4 system 
may only be able to deal with the aforementioned situations in a very limited way, 
such as by dropping objects or temporal occurrences of objects it is incapable of 
decoding or presenting, resulting in choppy audio-visual presentations. Further, 
MPEG-4 may not offer any sophisticated control by the user of those kinds of 
situations. To get around such limitations of the parametric system, one potential 
option for MPEG -4 development is in a programmatic system. 

Brief Summary Text (14) : 

The use of application programming interfaces (APIs) has been long recognized in 
the software industry as a means to achieve standardized operations and functions 
over a number of different types of computer platforms. Typically, although 
operations can be standardized via definition of the API, the performance of these 
operations may still differ on various platforms as specific vendors with interest 
in a specific platform may provide implementations optimized for that platform. In 
the field of graphics. Virtual Reality Modeling Language (VRML) allows a means of 
specifying spatial and temporal relationships between objects and description of a 
scene by use of a scene graph approach. MPEG-4 has used a binary representation 
(BIFS) of the constructs central to VRML and extended VRML in many ways to handle 
real-time audio/video data and facial/body animation. To enhance features of VRML 
and to allow programmatic control, DimensionX has released a set of APIs known as 
Liquid Reality. Recently, Sun Microsystems has announced an early version of 
Java3D, an API specification which among other things supports representation of 
synthetic audiovisual objects as scene graph. Sun Microsystems has also released 
Java Media Framework Player API, a framework for multimedia playback. However, none 
of the currently available API packages offer a comprehensive and robust feature 
set tailed to the demands of MPEG-4 coding and other advanced multimedia content. 

Brief Summary Text (16) : 

The invention provides a system and method for interfacing coded auidovisual 
objects, allowing a nonadaptive client system, such as the parametric MPEG -4 
system, to play and browse coded audiovisual objects in adaptive fashion. The 
system and method of the invention is programmatic at an architectural level, and 
adds a layer of adaptivity on top of the parametric system by virtue of a defined 
set of application programming interfaces specifically configured to access and 
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L8: Entry 3 of 4 



File: USPT 



May 2, 2000 



DOCUMENT-IDENTIFIER: US 6057884 A 

TITLE : Temporal and spatial scaleable coding for video ob j ect planes 



Abstract Text (1) : 

Temporal and spatial scaling of video images including video object planes (VOPs) 
in an input digital video sequence is provided. Coding efficiency is improved by 
adaptively compressing scaled field mode video. Upsampled VOPs in the enhancement 
layer are reordered to provide a greater correlation with the input video sequence 
based on a linear criteria. The resulting residue is coded using a spatial 
transformation such as the DCT. A motion compensation scheme is used for coding 
enhancement layer VOPs by scaling motion vectors which have already been determined 
for the base layer VOPs. A reduced search area whose center is defined by the 
scaled motion vectors is provided. The motion compensation scheme is suitable for 
use with scaled frame mode or field mode video. Various processor configurations 
achieve particular scaleable coding results. Applications of scaleable coding 
include stereoscopic video, picture-in-picture, preview access channels, and ATM 
communications . 

Application Filing Date (1) : 
19970605 

Brief Summary Text (2) : 

The present invention relates to a method and apparatus for providing temporal and 
spatial scaling of video images including video object planes in a digital video 
sequence. In particular, a motion compensation scheme is presented which is 
suitable for use with scaled frame mode or field mode video. A scheme for 
adaptively compressing field mode video using a spatial transformation such as the 
Discrete Cosine Transformation (DCT) is also presented. 

Brief Summary Text (3) : 

The invention is particularly suitable for use with various multimedia 
applications, and is compatible with the MPEG - 4 Verification Model (VM) 3.0 
standard described in document ISO/IEC/ JTC1/SC29/WG11 N1642, entitled " MPEG - 4 Video 
Verification Model Version 7.0", April 1997, incorporated herein by reference. The 
invention can further provide coding of stereoscopic video, picture-in-picture, 
preview access channels, and asynchronous transfer mode (ATM) communications. 

Brief Summary Text (4) : 

MPEG - 4 is a new coding standard which provides a flexible framework and an open set 
of coding tools for communication, access, and manipulation of digital audio-visual 
data. These tools support a wide range of features. The flexible framework of MPEG - 
4 supports various combinations of coding tools and their corresponding 
functionalities for applications required by the computer, telecommunication, and 
entertainment (i.e., TV and film) industries, such as database browsing, 
information retrieval, and interactive communications . 

Brief Summary Text (5) : 

MPEG - 4 provides standardized core technologies allowing efficient storage, 
transmission and manipulation of video data in multimedia environments. MPEG- 4 
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achieves efficient compression, object scalability, spatial and temporal 
scalability, and error resilience. 

Brief Summary Text (6) : 

The MPEG - 4 video VM coder/decoder (codec) is a block- and object-based hybrid coder 
with motion compensation. Texture is encoded with an 8. times. 8 DCT utilizing 
overlapped block -motion compensation. Object shapes are represented as alpha maps 
and encoded using a Content-based Arithmetic Encoding (CAE) algorithm or a modified 
DCT coder, both using temporal prediction. The coder can handle sprites as they are 
known from computer graphics. Other coding methods, such as wavelet and sprite 
coding, may also be used for special applications. 

Brief Summary Text (7) : 

Motion compensated texture coding is a well known approach for video coding. Such 
an approach can be modeled as a three-stage process. The first stage is signal 
processing which includes motion estimation and compensation (ME/MC) and a 2-D 
spatial transformation. The objective of ME/MC and the spatial transformation is to 
take advantage of temporal and spatial correlations in a video sequence to optimize 
the rate-distortion performance of quantization and entropy coding under a 
complexity constraint. The most common technique for ME/MC has been block matching, 
and the most common spatial transformation has been the DCT. However, special 
concerns arise for ME/MC and DCT coding of the boundary blocks of an arbitrarily 
shaped VOP. 

Brief Summary Text (8) : ^ 

The MPEG - 2 Main Profile is a precursor to the MPEG - 4 standard, and is described in 
document ISO/IEC JTC1/SC29/WG11 N0702, entitled "Information 

Brief Summary Text (9) : 

Technology — Generic Coding of Moving Pictures and Associated Audio, Recommendation 
H. 262, 11, " March 25, 1994, incorporated herein by reference. Scalability 
extensions to the MPEG - 2 Main Profile have been defined which provide for two or 
more separate bitstreams, or layers. Each layer can be combined to form a single 
high-quality signal. For example, the base layer may provide a lower quality video 
signal, while the enhancement layer provides additional information that can 
enhance the base layer image. 

Brief Summary Text (12) : 

Accordingly, it would be desirable to provide temporal and spatial scalability 
functions for coding of video signals which include video object planes (VOPs) such 
as those used in the MPEG - 4 standard. It would be desirable to have the capability 
for coding of stereoscopic video, picture-in-picture, preview access channels, and 
asynchronous transfer mode (ATM) communications. 

Brief Summary Text (13) : 

It would further be desirable to have a relatively low complexity and low cost 
codec design where the size of the search range is reduced for motion estimation of 
enhancement layer prediction coding of bi-directionally predicted VOPs (B-VOPs) . It 
would also be desirable to efficiently code an interlaced video input signal which 
is scaled to base and enhancement layers by adaptively reordering pixel lines of an 
enhancement layer VOP prior to determining a residue and spatially transforming the 
data. The present invention provides a system having the above and other 
advantages . 

Brief Summary Text (15) : 

In accordance with the present invention, a method and apparatus are presented for 
providing temporal and spatial scaling of video images such as video object planes 
(VOPs) in a digital video sequence. The VOPs can comprise a full frame and/or a 
subset of the frame, and may be arbitrarily shaped. Additionally, a plurality of 
VOPs may be provided in one frame or' otherwise be temporally coincident. 



httpV/westbrs: 9000^in/gate.exe?f^TOC8&state=v7sodn. 1 2 .3i&USERID=nel-hady&DBNA. . . 1 0/7/04 



Record Display Form 



Pages of 20 



Brief Suirnnary Text (16) : 

A method is presented for scaling an input video sequence comprising video object 
planes (VOPs) for communication in a corresponding base layer and enhancement 
layer, where downsampled data is carried in the base layer. The VOPs in the input 
video sequence have an associated spatial resolution and temporal resolution (e.g., 
frame rate) . 

Brief Summary Text (23) : 

In a specific application, the base and enhancement layer provide a picture-in- 
picture (PIP) capability where a PIP image is carried in the base layer, or a 
preview access channel capability, where a preview access image is carried in the 
base layer. In such applications, it is acceptable for the PIP image or free 
preview image to have a reduced spatial and/or temporal resolution. In an ATM 
application, higher priority, lower bit rate data may be provided in the base 
layer, while lower priority, higher bit rate data is provided in the enhancement 
layer. In this case, the base layer is allocated a guaranteed bandwidth, but the 
enhancement layer data may occasionally be lost. 

Brief Summary Text (24) : 

A method is presented for scaling an input video sequence comprising video object 
planes (VOPs) where downsampled data is carried in the enhancement layer rather 
than the base layer. With this method, a first particular one of the VOPs of the 
input video sequence is provided in the base layer as a first base layer VOP, e.g., 
without changing the spatial resolution. Pixel data of at least a portion of the 
first base layer VOP is downsampled to provide a corresponding first downsampled 
VOP in the enhancement layer at a temporal position corresponding to the first base 
layer VOP. Corresponding pixel data of the first particular one of the VOPs is 
downsampled to provide a comparison VOP, and the first downsampled VOP is 
differentially encoded using the comparison VOP. 

Brief Summary Text (26) : 

A method* for coding a bi-directionally predicted video object plane (B-VOP) is also 
presented. First and second base layer VOPs are provided in the base layer which 
correspond to the input video sequence VOPs. The second base layer VOP is a P-VOP 
which is predicted from the first base layer VOP according to a motion vector 
MV.sub.P. The B-VOP is provided in the enhancement layer temporally between the 
first and second base layer VOPs. 

Brief Summary Text (27) : 

The B-VOP is encoded using at least one of a forward motion vector MV.sub.f and a 
backward motion vector MV.sub.B which are obtained by scaling the motion vector 
MV.sub.P. This efficient coding technique avoids the need to perform an independent 
exhaustive search in the reference VOPs. A temporal distance TR.sub.P separates the 
first and second base layer VOPs, while a temporal distance TR.sub.B separates the 
first base layer VOP and the B-VOP. 

Brief Summary Text (28) : 

A ratio m/n is defined as the ratio of the spatial resolution of the first and 
second base layer VOPs to the spatial resolution of the B-VOP. That is, either the 
base layer VOPs or the B-VOP in the enhancement layer may be downsampled relative 
to the VOPs of the input video sequence by a ratio m/n. It is assumed that either 
the base or enhancement layer VOP has the same spatial resolution as the input 
video sequence. The forward motion vector MV.sub.f is determined according to the 
relationship MV.sub.f = (m/n) .multidot . TR. sub . B . multidot . MV. sub . P /TR.sub.P, while 
the backward motion vector MV.sub.b is determined according to the relationship 
MV.sub.b =(m/n) .multidot. (TR.sub.B -TR . sub . P) . multidot . MV . sub . P /TR.sub.P. m/n is 
any positive number, including fractional values. 

Brief Summary Text (29) : 
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Abstract Text (1) : 

The invented method involves the object-based processing of parts of video frames 
referred to as Video Object Planes using 2-D meshes, wherein the color and shape 
information associated with the Video Object Planes are assumed to be known at 
every frame and wherein each video object is processed independently. The invented 
method more particularly " involves utilization of the Alpha Planes, which contain 
shape information, in object-based design of an initial 2-D mesh, wherein an 
Alpha Plane is used to form a constraining polygonal mesh boundary, as well as in 
object-based tracking of mesh node points, wherein motion vectors of nodes on the 
mesh boundary are constrained so that these node points always lie along the Alpha 
Plane boundary, by means of restriction of the search space or back-projection, and 
mesh-based Video Object Plane mapping takes into account any differences between 
the mesh boundary and the Video Object Plane boundary. Such invented methods may be 
computer-implemented or computer-assisted, as by being coded as software within any 
coding system as memory-based instructions executed by a microprocessor, PC or 
mainframe computer, or may be implemented in hardware such as a state machine. 

Application Filing Date { 1) : 
19970702 

Parent Case Text (2) : 

This application claims priority from U.S. provisional patent application Ser. No. 
60/021, 0.93, filed on Jul. 2, 1996, the disclosure of which is incorporated hereby 
by this reference. 

Brief Summary Text (5) : 

[I] Y. Altunbasak, A. M. Tekalp and G. Bozdagi, "Two-dimensional object based 
coding using a content-based mesh and affine motion parameterization," IEEE Int. 
Conference on Image Processing, Washington D.C., October 1995. 

Brief Summary Text (9) : 

[5] L. Chiariglione, " MPEG and multimedia communications," IEEE Trans, on Circ. and 
Syst. for Video Technology, vol. 1, no. 1, pp. 5-18, February 1997. 

Brief Summary Text (12) : 

[8] Y. Nakaya and H. Harashima, " Motion compensation based on spatial 
transformations," IEEE Trans, on Circuits and Systems for Video Technology, vol. 4, 
no. 3, pp. 339-356, June 1994. 

Brief Summary Text (15) : 

[II] T. Sikora, "The MPEG - 4 Video Standard Verification Model," IEEE Trans, on 
Circ. and Syst for Video Technology, vol. 7, no. 1, pp. 19-31, February 1997. 

Brief Summary Text (16) : 

[12] G. J. Sullivan and R. L. Baker, " Motion compensation for video compression 
using control grid interpolation," Proc. ICASSP '91, vol. 4, pp. 2713-2716, May 
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1991. 

Brief Summary Text (18) : 

[14] C. Toklu, A. T. Erdem, M. I. Sezan and A. M. Tekalp, "Tracking motion and 
intensity variations using hierarchical 2-D mesh modeling, " Graphical Models and 
Image Processing, vol. 58, no. 6, pp. 553-573, November 1996. 

Brief Summary Text (26) : 

Briefly summarized, the invented method involves the object-based processing of 
parts of video frames referred to as Video Ob j ect Planes using 2-D meshes, wherein 
the color and shape information associated with the Video Object Planes are assumed 
to be known at every frame and wherein each video object is processed 
independently. The invented method more particularly involves utilization of the 
Alpha Planes, which contain the shape information, in object-based design of an 
initial 2-D mesh, wherein an Alpha Plane is used to form a constraining polygonal 
mesh boundary, as well as in object-based tracking of mesh node points, wherein 
motion vectors of nodes on the mesh boundary are constrained so that these node 
points always lie along the Alpha Plane boundary, by means of restriction of the 
search space or back-projection, and mesh-based Video Object Plane mapping takes 
into account any differences between the mesh boundary and the Video Object Plane 
boundary. Such invented methods may be computer-implemented or computer-assisted, 
as by being coded as software within any coding system as memory-based instructions 
executed by a microprocessor, PC or mainframe computer, or may be implemented in 
hardware such as a state machine. 

Drawing Description Text (3) : 

FIG. 2 illustrates object-based forward motion modeling versus frame-based forward 
motion modeling and frame-based backward motion modeling. 

Drawing Description Text (6) : 

FIG, 5 illustrates details of object-based motion estimation and motion 
compensation with a forward tracking mesh layer. 

Detailed Description Text (3) : 

Object-based video representations allow for object-based compression, storage and 
transmission, in addition to object-based video manipulation, such as editing. 
Object-based video compression methods are currently being developed in the context 
of the MPEG -4 standardization process [5, 11] . This disclosure describes methods 
for object-based video motion representation using forward tracking 2-D mesh 
layers, where one mesh layer is used for each object. 

Detailed Description Text (4) : 

Following MPEG -4 terminology [11], a "Video Object" (VO) refers to spatio-temporal 
data pertinent to a particular object and a " Video Object Plane " (VOP) refers to a 
two-dimensional (2-D) snapshot of a Video Object at a particular time instant 
(similar to a video frame) . Each VOP consists of a number of color components, for 
instance a Y, U and V component, as well as a shape component or "Alpha Plane", 
describing its shape and opacity. This data structure is depicted in FIG. 1. VOPs 
can be I, P or B type as in MPEG -1 and -2, which are previously adopted and 
published standards that are precursors to the developing MPEG -4 standard. Those of 
skill in the art will appreciate that such VOP types will be referred to herein as 
I-VOPs, P-VOPs and B-VOPs, respectively, corresponding to I-frames, P-frames and B- 
frames in the case of MPEG - 1 or -2 . Note that the Alpha Planes are herein assumed 
to be known for every VOP in the VO. In practice, the Alpha Planes can be obtained 
using for example chroma-keying. Note further, that different video objects may 
have been acquired with different cameras. On the other hand, different video 
objects may have been obtained from a single camera shot, by partitioning each 
frame into the constituent video object planes . A layered video representation 
similar to the data structure described above was discussed in [18] . 
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